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ABOUT THIS MANUAL 


This manual introduces the ABS Remote Access Data Laboratory (RADL) as a mode for accessing ABS 
Confidentialised Unit Record Files (CURFs). 


All users should familiarise themselves with the information in this manual before using a CURE in the 
RADL and refer to the information regularly while using CURFs. 
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WHAT ARE ABS CURFS? 


Confidentialised Unit Record Files (CURFs) are files of responses to ABS surveys or censuses that have 
had specific identifying information about persons and organisations confidentialised. CURFs contain the 
most detailed statistical information available from the ABS, and are of most use to researchers and 
analysts who wish to run their own statistical queries on the data. 


The Census and Statistics Act includes a legislative guarantee to respondents that their confidentiality will 
be protected. This is fundamental to the trust the Australian public has in the ABS, and that trust is in turn 
fundamental to the excellent quality of ABS information. Without that trust, survey respondents may be less 
forthcoming or truthful in answering our questionnaires. For more information, see ‘Avoiding inadvertent 
disclosure in published statistics' and 'Microdata' on our web page How the ABS keeps your information 
confidential. 


This legislation allows the Australian Statistician to release unit record data, or microdata, provided this is 
done “in a manner that is not likely to enable the identification of a particular person or organisation to which 
it relates.” The ABS ensures the confidentiality of the data by: 


e removing any information that may uniquely identify any individual such as name or address 
changing a small number of values and removing very unusual records 

controlling the detail available for all records on the CURF 

perturbing or randomly adjusting income data 

excluding some data items that were collected. 


These changes are made in ways that retain the value of the data for researchers. 
There are three different CUREF types: 


e Basic CUREFs are the least detailed CURF type. Users can access Basic CUREFs in their own 
environment (from a CD-ROM/DVD). Individual data items in Basic CURFs are likely to be reported at 
broader levels of their classification than in Expanded or Specialist CURFs. 

Expanded CURFs are accessible only in the secure RADL and ABS Data Laboratory (ABSDL) 
environments. They contain finer levels of detail than Basic CURFs. For example, where a Basic 
CURF might report an age classification in five-yearly increments, an expanded file may include data 
against each year. 

Specialist CURFs are customised to the clients requirement to serve specific approved statistical 
purposes, with the cost to produce them fully cost recovered from the client. They may include data 
items and levels of detail not otherwise released, or may be configured in other ways responding to 
particular user needs. Specialist CURFs are only available for access in the ABSDL. 


More information about the CURF types and modes of access is available on the CURF Access Modes and 
Levels of Detail web page. 
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WHAT IS THE RADL? 


The Remote Access Data Laboratory (RADL) is a secure online data query service that approved clients 
access from the ABS website. Within the RADL users submit queries in SAS, SPSS or Stata analytical 
languages against CURFs that are kept within the ABS environment. The results of the queries are 
automatically checked then made available to the users. Because the CURFs are kept in the ABS 
environment, the ABS is able to release more detailed CURFs in the RADL than are available on CD-ROM. 


Key features of the RADL include: 


available 24 hours a day, 7 days a week 

e access from any computer with internet access, using your own user name and password 
analytical software (SAS, SPSS and Stata) provided within the system so you don't need your own 
licence 

e batch processing with a quick turnaround 

e personal workspace to manage your analysis and saved datasets. 


Restrictions on using the RADL include: 


e users are bound by the undertaking signed by the head of their organisation and the terms and 
conditions agreed by each user 

e user activity is monitored via regular RADL audits 

e some restrictions are applied to queries, and some SAS, SPSS and Stata commands are banned 

e some restrictions are applied to the size of the outputs returned to the user. 


For further information about the type of restrictions put in place by the ABS, see Rules and restrictions for 
use in this manual. 
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RADL USER REQUIREMENTS 


Before you start using the Remote Access Data Laboratory (RADL), you should read and understand this 
manual and the Responsible Use of ABS Microdata Guide. The user guide explains the specific 
responsibilities of microdata users and provides guidelines on the secure management of microdata. 


RADL users should familiarise themselves with the information linked in the left hand navigator of the 
RADL home page, including: 

e CURF reference documentation: contains important information about each CUREF including the 

technical manual, referencing information, frequency and weighting report 

Information for all RADL users: contains general news and information on any recent changes to 

the RADL 

e ABS alerts: contains important information and announcements for RADL users which should be 
checked every time you log into the RADL. Occasionally the ABS will conduct routine maintenance 
which may affect RADL access, which will be announced here. 


RADL users must: 

have been approved by ABS to use the CURF they wish to access in the RADL 

be able to write programs in one of the analytical languages in the RADL (SAS9.1, SPSS11.5 and 
Stata10). 


The ABS does not provide a code advisory service, so familiarity with SAS, SPSS or Stata is important. Log 
files can be used to determine problems in obtaining output. Further information is available in the RADL 
Help chapter of the User Manual: ABS Remote Access Data Laboratory (RADL). If you encounter any 
problems accessing or using the RADL, email the RADL Administrator. 


The ABS privacy policy outlines how the ABS handles any personal information that you provide to us. 
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AVAILABLE SOFTWARE IN THE RADL 


The RADL includes analytical software for the SAS, SPSS and Stata statistical languages. This software is 
provided in RADL so no licence is necessary in the user's own environment. The RADL contains the 
following versions and modules: 


SAS 
RADL uses SAS version 9.1, SAS modules available in RADL are: 


e¢ SAS/BASE 
e¢ SAS/STAT 


SPSS 
RADL uses SPSS version 11.5. SPSS modules available in RADL are: 


e SPSS Base 
e SPSS Regression Models 


Stata 
RADL uses Stata version 10 special edition. Stata modules available in RADL are: 


e Stata Special Edition software 
e Stata modules hnblogit and ivreg2 
e the user-written svr suite of commands for estimation and analysis using replicate weights 
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CONNECTING TO THE RADL 


To connect to the RADL, users can either type www.abs.gov.au/radl into the address field of their web 
browser or go to the Remote Access Data Laboratory web page and click the green 'Access RADL' button. 


A warning message may appear. The message indicates that the user is entering a secure web site. Press 
OK to go to the RADL log-in screen. 


Log in using your user name and password supplied by the ABS. The password is case sensitive. Once 
logged in, the RADL Home page opens. If you do not already have a RADL user name and password from 
the ABS, you will need to apply for access to CUREFs in the How to Apply for CURFs web page. 


Information on submitting a new request (job) using a CURF can be found in the Submitting a request in the 
RADL section of this manual. 


To log out of the RADL, select 'Log out’, either from the navigator on the RADL Home page or from the links 
at the bottom of most screens. To ensure you are fully logged out of the RADL you should also close your 
browser at the end of your session. If your RADL session is idle for 30 minutes, any jobs which have not 
been fully submitted will need to be recreated. You will be prompted to log in again before you are able to 
resume working in the RADL. 
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USER RESTRICTIONS 
When using the RADL, users must comply with the following key obligations: 
Access 
e users must store their user ID and password securely 


e users must not allow others to access or use their password, even if they have also been approved for 
access to the same CURF. 


Code 


e all datasets of unit records created by the user must include the ABS unit identifiers 

e certain program statements must not be used in the RADL. For information on banned commands, 
see the Restrictions for program statements chapter of this manual 

e certain program options must be retained in the RADL. For information, see the Restrictions for 
program statements chapter of this manual 


Output 


e users are not permitted to output the entire contents of a CURF, or output subsets of the CURF 
cumulatively over time outside the RADL 

users are not permitted to output large amounts of unit record data, or combine unit record output 
cumulatively over time outside the RADL 

outputs will only be returned automatically if the output does not exceed list or table limits and total 
output size limits 

output released as Keep Secure is considered too detailed for general release and will be provided in 
a separate Keep Secure output file. Keep Secure output cannot be shared with other people unless 
they also have been approved by the ABS to access that CURF. Keep Secure output must be stored 
securely or destroyed after viewing. For more information on how to store microdata securely see the 
Secure storage of ABS microdata section of the Responsible Use of ABS Microdata guide. 

particular combinations of sub-state geographic data items are not permitted to be used in the 
generation of analytical outputs. For more information, see the Restrictions for geographic variables 
section of this manual. 


RADL users must cooperate with any ABS directions in respect to their access, code and output. 
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RETENTION OF ABS IDENTIFIERS 


Users must ensure that they retain the ABS unit identifiers. This applies to temporary datasets as well as 
users' saved datasets. This enables the automated checking facilities in the RADL to perform confidentiality 
checks, and assists in the automated clearance of output. Before output is returned to users, RADL 
automatic checks are applied during the execution of users' submitted code. The checks look for code 
segments that have inadvertently dropped (or failed to keep) all ABS unit identifiers. If these code segments 
are found, a message will be printed to the log or output file. 


Where a dataset contains more than one ABS unit identifier, all of these must be retained, even though 
some of them may not be needed for your analysis. A list of unit identifiers can be found for each CURE in 
the 'CURF reference documentation link’ in the navigator of the RADL. 


If a unit identifier is removed from a temporary or saved unit record file, the following error message will be 
displayed: 


SAS 
The program will terminate as a "Failed" job, with a message appearing in the log file. 


<ABS> ERROR: ABS ID variable ABSHID dropped from input dataset Xxxx.xxxxx. 
<ABS> ERROR: ABS ID variable ABSPID dropped from input dataset Xxxx.XXxXxx . 
<ABS> ERROR: RADL pre-step dataset check failed. This job will now be aborted. 


SPSS 
The program will be terminated, with a message appearing in the log file. 


The unit identifier field(s) ABSFID ABSHID ABSIID ABSJID 
were dropped, contrary to RADL rules. This job will be stopped. 


Stata 
The program will continue to run, with unit identifiers being automatically retained. A message will appear in 
the output file. 


ABS protection does not allow dropping of the following variable(s): ABSHID 
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OUTPUT AND TABLE RESTRICTIONS 
Restrictions apply to the amount of data that may be released as output through the RADL. 


Unit record data 
If the output contains unit record data, the following restrictions apply: 


Number of Number of 
variables records 
allowable 


ONOTARWNHE 
(o>) 
oO 


If these limits are exceeded, the output will not be produced and the log file will contain a message stating 
that the procedure exceeds the ABS unit record count limit and will not be output. While the unit record 
identifiers must be retained in user's saved datasets, the identifiers do not need to be included in output 
files. 


Aggregate data 
If the output does not contain unit record data, the following restriction applies: 


e¢ no more than 1,000 cells (i.e. multiply the number of output categories per variable) may be output. 


If this limit is exceeded, the job will be executed but the output will be directed to a file which will be withheld 
from the user. The following message will be written to the log: ‘The <procedure name> step output which 
would have appeared here exceeded ABS output size limits’. Users wanting to access the withheld output 
must request the file from the ABS. See the Requesting withheld output if the status of your RADL job is “On 
Hold” section of this manual. 


Frequency counts and cross-tabulations are subject to further checking. This checking will look for small 
contributor cells. The table is also assessed for sparseness. Both issues are considered when deciding if 
the table can be released. 


Total output 
The total output in a particular job (excluding any output already being withheld) will be restricted to the 
following: 


¢ no more than 100 different unit records may be output 

e no more than 8 variables may be output for any one unit record 

e no more than 2,000 cells (i.e. multiply output categories per variable) of aggregate data may be output, 
or 

e no more than 300,000 bytes in size may be output. 
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RESTRICTIONS FOR PROGRAM STATEMENTS 


The RADL incorporates some automated restrictions that disable some functions or commands within SAS, 
SPSS and Stata. Other options within SAS, SPSS and Stata must be retained. 


All languages 


e Graphical output is not allowed in the RADL. All program statements creating graphical output 
are not permitted. 

e Particular combinations of sub-state geographic data items are not permitted to be used in the 
generation of analytical outputs. For more information, see the Restrictions for geographic 
variables section of this manual. 


SAS 
The following SAS options must be used. Users are not required to turn on these options as the RADL 


system does it automatically. However, users should be aware that the RADL will not run their code if these 
options are turned off. 


e mprint e source2 
e source e symbolgen 


Users should not use the following statements. The RADL will generate them automatically. 
e filename e libname 


The following SAS commands, functions and macros are not allowed: 


e | e¢ modulein ¢ proc chart ¢ proc power 
e %ds2csv e¢ modulen ¢ proc corr with (plot 
e add e nomprint plots= option subcommand) 
e bye e nosgen e proc document e proc prt 
e dcreate ¢ nosource © proc export e proc prtdef 
e endsas e nosource2 e proc gplot e proc prtexp 
e filename e nosymbolgen e proc femp e proc surveyfreq 
e libname e ods ¢ proc glmpower ¢ proc template 
e¢ module e pathname (plot e sysexec 
e¢ modulec e peek subcommand) e system 
e¢ modulei e peekc ¢ proc import e sysuserid 
e¢ moduleic e poke e proc infomaps ex 
e proc plot 


Note that the automated checks for these items currently prevent the use of hexadecimal strings such as 
("1F4C"x) although you can use hexadecimal numbers such as 1F4Cx. 


SPSS 
The RADL specifies values for the following settings: 


e box e¢ mexpand 
e errors e mprint (“set" 


e journal 
e¢ messages 


statement) 
e printback 
e results 


Users should not use the following statement. The RADL will generate the file handle automatically. 


e file handle 


The following SPSS commands and functions are not allowed: 


lenddefine 
Inoexpand 
loffexpand 
cluster 
define 
finish 


e include 

e print (without 
the outfile= 
option) 

e print eject 
(without the 


outfile= option) 


print space 
proximities 
report 
script 


e uncompressed 
(on save or 
xsave) 

e write (without 
the outfile= 
option) 

e write formats 


Note that the PRINT FORMATS and WRITE FORMATS statements are permitted. 


Stata 


Users should not use the following statements. The RADL will generate them automatically. 


e adopath 


e set memory 


The following Stata commands and functions are not allowed: 


#delimit 
ado 
adosubdir 
bootstrap 
browse 
brr_hadamard 
matrixfile 
capture 
cc 

cchart 
CCl 

clear 
collapse 
contract 
copyright 
corrgram 
cs 

csi 
cumsp 
cusum 
diagplots 
db 

dir 
display 
do 

doedit 
dotplot 
dstdize 
edit 

exit 
fdasave 
e fdause 

e fdades 


e findfile 

e for 

e foreach 
e forvalues 
gettoken 
global 
graph 
gr7 
graph7 
greigen 
grmeanby 
help 
hexdump 
histogram 
istdize 

ir 

iri 

jknife 
kap 
kdensity 
line 

local 

log 

lv 

macro 
matrix 
mcc 
mcci 
mhodds 
more 
net 
news 
noisily 


pathsearchlist 
pchart 
pergram 
personal 
plot 

pr 
program 
quietly 
rchart 
renpfix 
reshape 
roccomp 
rocfit 
rocgold 
rocplot 
roctab 
run 
serrbar 
set 

set trace on 
shell 
shewhart 
shriek 
sleep 
spikeplot 
stack 
stem 
stcoxkm 
stcurve 
stphplot 
sts 
sunflower 
survwgt 


svrtab 
svymean 
svyprop 
svyratio 
svytab 
svytotal 
syntax 
sysdir 
sysuse 
tab1 
tab2 
tabi 
tabodds 
tabstat 
twoway 
type 
update 
version 
view 
viewsource 
webuse 
while 
window 
winexec 
wntestb 
xchart 
xcorr 
xmlsave 
xmluse 
xpose 
xttab 
xttrans 


Stata commands that produce protected tabular data that are allowable in the RADL include: 


e table ¢ svrmean 
e tabulate (ta) e svrtotal 
e summarize (Su) _—e svrratio 


Stata commands that are allowed but are subject to limits: 


e list 
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RESTRICTIONS FOR GEOGRAPHIC ANALYSIS 
The following restrictions apply to Expanded CUREFs only and do not need to be applied to Basic CURFs. 


Geographic data items vary between Expanded CURFs but typically include some or all of the following: 
State, Capital city/balance of state, Section of state, Remoteness and Socio-economic Indexes for Areas 
(SEIFA). The remaining variables on the CURF can be referred to as non-geographic data items. As Capital 
city/balance of state, Section of state, Remoteness and SEIFA split geographic areas (typically States/ 
Territories) into finer areas they are referred to as sub-state geographic data items. Particular combinations 
of sub-state geographic data items are not permitted to be used in the generation of analytical outputs, 
whether that output be printing a small number of observations, producing a table or undertaking more 
complex statistical analyses such as modelling. 


The following information provides a more detailed guide to the use of sub-state geographic data items. 


Printing_(to screen)_a limited number of records 


e When printing unit records incorporating geographic variables only, there are no restrictions on 
the number of sub-state geographies that can be used. 

e When at least one non-geographic data item is included in the list of variables then users are not 
permitted to include more than one sub-state geography. 


Tabular outputs 


e Users are permitted to use unlimited geographic data items in tabular output where no non- 
geographic data items are output (these are referred to as ‘simple’ tables). 

e Tabular output containing non-geographic data items are permitted to contain a maximum of one 
sub-state geography. 


Analytic outputs 


e Users are permitted to use multiple sub-state geographies in some types of analysis where they 
are not cross-classified with other sub-state geographies. As an example, users are permitted to 
perform a regression analysis with multiple sub-state geographies as explanatory variables, but 
would not be permitted to include interaction terms consisting of multiple sub-state geographical 
data items. 


Filtering_or subsetting datasets 


e Filtering or subsetting datasets by more than one sub-state geographic data item is not 
permitted. Additionally, any filtering or subsetting of datasets using one sub-state data item 
constitutes the use of that data item in any subsequent output or analysis. For example, if a 
dataset has been subset using a particular sub-state geographic data item then any record prints 
containing non-geographic data items are not permitted to output any further sub-state 
geographies. Likewise, any tabular output containing non-geographic data items produced from 
the subset dataset are not permitted to contain any further sub-state geographies. 


Deriving_new variables 


e No new variables are permitted to be derived from the combination of more than one sub-state 


geographic data item. 
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RADL ACTIVITY CHECKS 


Checks of jobs submitted to the RADL are conducted throughout the year by the ABS. The purpose is to 
identify any confidentiality issues within users' code, log or output files. Confidentiality issues may arise 
through users failing to follow, accidentally or otherwise, restrictions or rules for using the RADL. 


For each job that is audited, the code, log and output files are reviewed by ABS staff. If misuse of ABS data 
is identified, the user will be notified and the matter discussed. Users are required to cooperate with any 
ABS directions with respect to their code and output. 


Any breach of the CURF Undertaking, may result in withdrawal of service to individuals and/or 
organisations. Further information is contained in the Consequences of Failing to Comply web page. 
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SUBMITTING A REQUEST 


To interrogate a CURF, a request (code) must be submitted. All requests are run in batch mode. Interactive 
(or point-and-click) processing is not available in the RADL. 


After connecting to the RADL, submit a request by clicking on the 'Submit new request’ link from the 
navigator on the RADL Home page. This will take you to the New Microdata Request form which contains 
the following fields: 


Job name: enter a name for your job, this will help you recognise the job when checking your output 
Description: enter a description for your job, this is not a mandatory field and can be left blank 

Language (drop-down box): select the statistical language that will be used. Note the Innovation languages 
must be used with the Innovation CURF 

Microdata: select the CURF dataset that you will be using 

Source code: source code can be submitted via a plain text file (.txt), a SAS (.sas) file or a Stata (.do) file 
saved to your computer. This is done by clicking on the 'Browse...' button and locating the code file that will 
be used. Alternatively you can manually enter your code by clicking on the 'Enter Code...’ button 

Estimated execution time: jobs should be run as normal (< 10 minutes) in most cases. If your jobs are timing 
out or you are running complex modelling, select Long (>=10 minutes) 

Notify by email when complete? select 'yes' if you want to receive an email notifying you once your job has 
completed 

Submit: when all fields have been completed click ‘submit’. 


Note that all fields marked with an asterisk on the form are mandatory. 


SIRI Y Te 

Australian 
Bureau of 
Statistics 


Fields marked * are mandatory fields. 


* 


Job name: 
Description: 
Language: 
Input: 


Microdata: 


Source code: 


Location of file containing code: 


OR 
Enter code: 


Estimated execution Time: 


Notify by email when complete?: 


New Microdata Request (Reference: 0135) 
Status: New Request Go back 


Submitted: Today 11:40 AM 
Retention Period: 14 (days) 


Examine Dummy CURF data 


SAS v 


Population Census HSF 2001 Expanded 
Post Production Test 
Pregnancy and Employment Transitions Sure 
RADL Test CURF 2007 


|) | 


Browse... 


® Normal (< 10 minutes} 
OLong (== 10 minutes) 


@®No 
OYes at) Yourname @ Your work 


Once all the mandatory fields have been completed, the user can submit their request for processing by 
pressing Submit. If the confirmation screen has not been turned off in the user's personal profile the 


following screen will appear. This screen confirms the details of the job that is about to be submitted. If the 


request is correct press OK, otherwise press Cancel to return to the request. 


Message from webpage xs 


e CURF Request 3412 


Job Name: Examine Dummy CURF data 
Description: 

Language: SAS l 
Input: RADL Test CURF 2007; 


Source Code: 
Source code entered 


Store Results in: results3412.zip 
Estimated Execution time: Normal (< 10 minutes) 
Notify by Email when Complete to: radI@abs.gov.au 


Press OK to submit this request, Cancel to return to the request 


If you do not wish to see this screen again, change the ‘Confirmation 
screen’ setting in your Profile 


| OK | | Cancel 


Once a request has been submitted (i.e. by pressing OK), the user will be directed to a new screen which 
provides a number of options, including submitting another request, checking the status of previous 
requests, and logging out of the RADL. Each option can be activated by clicking on the relevant underlined 
text. If the submission screen has been turned off you will be taken to the screen which shows your requests 
sorted by job name. For more information see the Managing your user profile section of this manual. 


Status: this field is updated as the status of the job changes. For example, New Request, Complete, Failed, 
etc. Press refresh in your browser to update the status 

Requestor: this field refers to the person who submitted the job. Users can only see their own jobs 
Submitted: this field states the date and time that a job was submitted 

Retention Period: refers to how many days a submitted job will be accessible via the RADL. Users can 
modify the retention period in a number different ways. For more detail on the ways to modify the retention 
period see the Changing job retention periods section of this manual. 


Each request submitted will be allocated a unique reference number. This is used to assist in the 
identification of jobs, for example, Reference: 0020. This reference number is appended to the end of all file 
names produced by the RADL system, for example, output0020.txt. 
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Referencing a CURE in your code 

When submitting a job users are required to select the CURF(s) which will be accessed by that job. The 
RADL system requires that you reference each dataset (level) of the CURF that you want to use. The 
following is an example for the General Social Survey 2010 Expanded CURF: 


Language 


Code example for 
referencing CURF 


Further information 


SAS 


SET 
GSS10E.GSS10EP 
SET 
GSS10E.GSS10ESD 


A ‘libref will be assigned by the 
RADL to the SAS datasets. The libref 
can be used by clients to reference 
the datasets, so SAS similar 
commands can be used in your 
RADL code. The SAS database also 
contains a ‘format’ catalogue which 
contains formats which have been 
applied to the variables in the 
datasets. This formats library will be 
automatically assigned by the RADL 
for any code using this CURF. 


Stata 


use "GSS10EP" 
use "'GSS10ESD" 


Macro variables containing the path 
names to the CURFs are 
automatically generated in the 
RADL. 

When using these references take 
care to use the correct characters for 
the single quotes. 


SPSS 


GET FILE=GSS10EP. 
GET 
FILE=GSS10ESD. 


A ‘file handle’ will be assigned by the 
RADL to the SPSS datasets. These 
file handles can be used by clients to 
reference the datasets, so SPSS 
syntax similar to the example can be 


used in your RADL code. 


For the complete set of file references see the 'CURF reference documentation’ link in the navigator of the 
RADL. 


In order to maintain the integrity of the CURFs users are given read access to the CURF files - CURFs 
cannot be over-written. Given this restriction if you want to modify the CURF you will need to save a copy of 
the CURF. For more information about saving CURFs in the RADL see the How to save files to your 
personal workspace section of this manual. 


Using more than one CURE together 

It is possible to use more that one CURF together in your analysis. However, you may only merge 
aggregate data to unit record data, or aggregate to aggregate data between the CURFs. You must not 
merge unit record to unit record data across CURFs. 


To merge CUREFs, in the New Request screen, select more than one CURF in the Microdata window by 
holding down the Control button on your keyboard. You will also need to reference both CUREFs in your 
code. Alternatively your code can reference a saved dataset from one CURF and reference another CURF 
that you have selected in the Microdata window. 


There are certain CURFs that cannot be used together. These CURFs are provided to users under a 
separate user ID so that they cannot be combined. These additional user IDs will end with "_INNOV", 
"STATE", "_DETAIL", or "_ REM". 
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SAMPLE CODE FOR GETTING STARTED 


Introductory RADL sessions for SAS, SPSS and Stata are available in the ‘Information for all RADL users' 
section of the RADL. The information is intended to introduce users to the RADL system and using code to 
produce basic tables and statistics. 


Additional code examples provided in this manual are linked below: 


e displaying a complete list of variable names 

producing aggregate data in Comma Separated Value format 
cross-tabulations 

merging levels within a CURF 

creating ranges 

saving files and using saved datasets 

using File management commands. 
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Users can save SAS, SPSS and Stata files in their personal workspace within the RADL. This workspace is 
accessible for all the jobs you submit, allowing you to save and reuse files across sessions. Users are not 
able to access files outside their workspace and cannot share saved dataset with other users. It is not 
necessary to specify the location when saving files. To manage the files that you have saved to your 
workspace, refer to the File management chapter of this manual. 


Procedures for saving and reusing files are: 


SAS 

A library reference of 'saved' will be created for all SAS jobs. This can be used for saving derived SAS 
datasets to your workspace and for accessing previously saved datasets. To save a SAS output dataset to 
your personal workspace use: 


data saved.MYSAS (keep = ABSHID ABSPID age sex income); 
run; 


To retrieve this dataset in a subsequent RADL job use data=saved.MYSAS. For example, to create a 
frequency table from this saved dataset use: 


proc freq data=saved.MYSAS; 
tables age*sex; 
run; 


SPSS 

A library reference of 'saved' will be created for all SPSS jobs. This can be used for saving derived SPSS 
datasets to your workspace and for accessing previously saved datasets. To save an SPSS output dataset 
to your personal workspace use: 


save outfile = saved’MYSPSS’. 
To retrieve this dataset in a subsequent RADL job use: 


get file=saved’MYSPSS’ 
Note: the single quote marks must be used. 


Stata 

A macro variable is automatically generated that helps you reference files you want to save and reuse in 
later jobs. The macro SAVED (in uppercase) points to your personal data store. Within RADL this macro 
resolves to a text string that contains the path name to your personal workspace. When you save a dataset 
you need to refer to it using SAVED as a prefix. Since the path name may contain spaces, it is best to put all 
RADL macro invocations in double quotes. To save a dataset use the command: 


save " “SAVED'MYSTATA" 
To retrieve this dataset in a subsequent RADL job use: 


use " “SAVED'MYSTATA" 


Note: when using this workspace reference take care to use the correct characters for the single quotes. 
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To check the status of a previously submitted job, select one of the 'View requests by' links from the 
navigator on the RADL home page. Submitted jobs can be viewed by Status, Date, Job name or by CURF. If 
the submitted job is not listed on the screen, press the Refresh button on the browser to update the screen. 
It may take a few minutes for a new job to be displayed. 


The first link displayed is a copy of the request that was originally submitted and contains the job name 
given to the request. This file contains the original code that was included in the request. 


Results: The second link entitled ‘Results’ contains the output and log files generated from the request. All 
the results documents have been zipped into one file entitled results#.zip that can either open or save 
depending on the settings of the software used. If the job has successfully completed the results#.zip may 
contain: 


e output# txt: this output has low confidentiality risk and is considered safe for sharing or publishing. By 
opening the output#.txt users can view the output from the job including statistical procedures and 
listings of temporary datasets that do not contain unit record data. 

keep _secure#.txt: this contains output too detailed for general release. This output must not be shown 
to anyone who does not have access to the CURF and the results cannot be published without 
permission from the ABS. By opening the keep_secure#.txt (if present), users can view output from 
their SAS, SPSS and Stata jobs which contains too much detail for general release. The amount of 
such data that may be returned is small. If users try to output more than a predefined maximum 
amount of data this step will not be executed, although the rest of the job will still run. For further 
information see the Rules and restrictions for use section of this manual. 

log#.txt: this file contains a detailed record of how your job was run. This file may include RADL 
messages about output that has been withheld or denied. It may also contain details about errors if the 
job did not run successfully. If all the output was withheld, only the log file will be produced. The log file 
users receive back from the ABS has been processed by the RADL system. This may limit information 
available for debugging of code (if it did not run as expected). The ABS has provided a list of 
frequently asked questions, sample code and introductory sessions in SAS, SPSS and Stata which 
may also help users to debug code. 

File ManagementLog#.txt: file management output is titled file managementLog#.txt. For more 
information see the File management chapter of this manual. 


Validation: To assist users to validate their results the ABS provides frequency and weighting reports 
specific to each CURF. These are available in the 'CURF reference documentation’ link in the navigator of 
the RADL and have been made available for CURFs released after September 2003. 


Withheld output: If the log file or output files indicate that data was withheld for any reason it is possible to 
request that output be released. To request the release of withheld output, select the link ‘Request Withheld 
Output' in the results screen. Further information is contained in the Requesting withheld output if the status 
of your RADL job is “On Hold” section of this manual. 
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REQUESTING WITHHELD OUTPUT IF THE STATUS OF YOUR RADL JOB IS "ON HOLD" 


If the log file or output files indicate that data was withheld for any reason it is possible to request that output 
be released. Denied output will not be released. To request the release of withheld output, go to the results 
screen and select the link ‘Request Withheld Output’. 


Another screen will appear, requiring the following information: 


e name of the file or files. More than one can be requested as long as it all relates to the one job 
(as indicated by the job number). The file name will be included in the log (example hold5.txt) 

e a description of the outputs included in the file 

e any further relevant comments. 


Once this screen is completed, click OK. An automated email is sent to the RADL Administrator requesting 
the file. Users will be contacted within one working day about the progress of their request. 
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EDITING YOUR PERSONAL PROFILE 
To edit your personal profile click on the Your Profile link from the navigator on the RADL Home page. 


To change information or options on this page, click on the Edit link at the bottom of the page (or double- 
click anywhere on the screen). Information that can be changed includes: 


address 

phone number 

email address 

organisation 

data retention period 

confirmation screen (display preference) 
submission screen (display preference) 


Once the appropriate changes have been made, click on the Save and Exit link at the bottom of the screen. 
If the changes are not required to be saved, click on the Exit link. Note that users cannot change Name and 
Available ABS Data. If either of these fields are not correct, contact the RADL Administrator at 
radi@abs.gov.au. 
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CHANGING JOB RETENTION PERIODS 


The retention period refers to how many days a submitted job will be accessible from the RADL. The default 
retention period is set in the user's profile. The initial default retention period is 7 days, but this can be 
modified according to user's needs. Setting the retention days to zero will keep jobs indefinitely. 


The retention period can be amended in several different ways, in the: 


Submit new request screen (for a new job): when submitting a new job the retention period field in the New 
Request screen can be edited. This change to the retention period relates to this job only. 

View requests by link (for a previously submitted job): when checking a previously submitted job, the user 
can change the retention period by going back into the request document. Once in the request document, 
select the link Change Retention Period. Enter the new retention period and select OK. This will modify the 
retention period for this job only. 

Your Profile: in the personal profile, users can edit the default retention period for all new requests. 
Changing the default retention period in the personal profile will affect all new requests submitted after this 
change. It will not change the retention period for previously submitted jobs. 
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WHAT IS FILE MANAGEMENT? 


RADL users can save files when running jobs in the RADL. Saved files can then be used in subsequent 
RADL jobs. Saved files are stored in the user's personal workspace in the RADL and can be managed using 
the File Management facility. 


To access the File Management facility use the New Request screen in the RADL. The procedure to submit 
a File Management request is identical to submitting a request relating to a CURF using SAS, SPSS or 
Stata with the following exceptions: 


e Select File Management as the language for your program (instead of SAS, SPSS, Stata or Innovation 
CURF versions of the languages) 


e No input Microdata is required to be selected for a File Management job 
e Specific File Management commands must be used to manage data holdings. These are outlined in 
the File management command code section of this manual. 


This facility allows users to obtain a directory listing of their data holdings, and to rename or delete files. 
Users cannot open or access the data directly from the File Management facility. Information on how to save 


and use saved datasets is outlined in the How to save files to your personal workspace section of this 
manual. 


To ensure the RADL runs smoothly we rely on users to delete saved datasets that they no longer need. 
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FILE MANAGEMENT COMMAND CODE 


The File management facility within the RADL supports three simple commands, and also allows a user to 
include comments or blank lines in their File Management code. Click on the Enter Code... button to enter 
your File Management commands. Each File Management command is entered as text and must be 
entered with one command per line. The functions available are: 


Directory Listing: to obtain a listing of all RADL files, users must use the dir (directory or list) command. This 
command has no parameters, i.e. enter the command name by itself. Users will receive a directory listing in 
their job output. 


dir 


Deletion of Files: to delete any RADL files, users must use the del (delete) command. The del command is 
followed by one or more specifications of files to delete. Each specification may be an explicit file name or 
may contain wildcards as for DOS (e.g. *). 


del oldstuff.txt (This deletes the file called oldstuff.txt) 
del *.tmp (This deletes all files ending in .tmp) 
del other.* (This deletes all files called ‘other’ irrespective of their file type) 


Renaming Files: should users wish to rename any of their saved datasets, they must use the ren (rename) 
command. The ren command is followed by the current name of the file and then the new name the user 
wishes to assign to it. Special characters may not be used. 


ren myjob.txt keepit.txt (This renames a file from 'myjob.txt' to 'keepit.txt’) 


Adding_comments: users may include comments into their code with the other File Management commands 
(dir, del or ren) when submitting File Management jobs. Blank lines may also be used to make the code 
more readable. Comments in the File Management facility, like other commands, must be entered as single- 
line statements. To put include a comment in the File Management job, start the comment line with either 
comment, rem (remark) or * followed by a space. 


comment Deleting old files (This includes a comment ‘Deleting old files' within a file management job) 
rem Checking my files (This includes a comment ‘Checking my files’ within a file management job) 
* Updating file names (This includes a comment ‘Updating file names’ within a file management job) 
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BEGINNING YOUR ANALYSIS 


What should | do before | start using RADL? 

Where can | find information about the CURF and data items? 
Why can't | see my approved CURFs in the RADL? 

How do | get a complete list of variable names for my CURF? 
How do | validate my results? 


What should | do before | start using the RADL? 
Prior to submitting your code and commencing your analysis, RADL users must: 


e have been approved by ABS to use the CURF they wish to access in the RADL 

e have familiarised themselves with the Technical Manual and any Information Papers relating to the 

CURF 

have read the User Manual: ABS Remote Access Data Laboratory (RADL) 

have read the User Manual: Responsible Use of ABS CURFs 

e be able to write programs in one of the analytical languages in the RADL (SAS9.1, Stata10 or 
SPSS11.5), as ABS does not provide a code advisory service 

e know how to use an internet browser 

e be able to use a zip program (WinZip or equivalent programs). 


Where can | find information about the CURF and data items? 

Technical Manuals or Information papers providing information regarding the data items and variables for 
each CURF are available in the 'CURF reference documentation’ > 'CURF documentation’ section of the 
RADL. These documents are CURF specific and should be read before using a CURF as well as referred to 
regularly while using a CURF. 


Why can't | see my approved CURFs in the RADL? 
There are three possible reasons: 


e Check that you have completed all steps in the New Request screen. For further assistance see the 
Submitting a request in the RADL section of the User Manual: ABS Remote Access Data Laboratory 
(RADL) 

e For some CURFs, additional RADL user IDs are allocated to approved users. Some users will have 
user identifiers ending with "_INNOV", "_STATE", "_DETAIL", or "_REM". If you have one of these user 
IDs you should log in with each of your user IDs to check whether the CURF you want to use is 
attached to your other ID 

e Your application for access has not yet been finalised. Check in MiCRO to see the status of your 
CURF application or contact microdata.access@abs.gov.au or call 02 6252 7714 for assistance. 


If this doesn't fix the problem, contact the RADL Administrator for assistance by emailing radli@abs.gov.au 
or call 02 6252 7714. 


How do | get a complete list of variable names for my CURF? 
A full list of variables and their short names can be obtained: 


e from the 'CURF reference documentation’ section of the RADL 
e via the Data Item List linked from the Expected and available Microdata web page (click on the CURF 
link, go to the Downloads tab) 


e by running a job to display the variables within the referenced dataset. Below are some examples of 
the code required to run these jobs. Note that you will need to run one of these for each level within a 
CURF. 


SAS 
proc contents data = LIBNAME.CURFLEVEL1 ; 
run; 


SPSS 

get file = CURFLEVEL1. 
display variables. 
execute. 


Stata 
use "CURFLEVEL1 ™ 
describe 


How do | validate my results? 
The 'CURF reference documentation’ > 'CURF documentation’ section of the RADL contains frequency and 
weighting reports. These reports have been made available to assist users in validating their results. 
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MANDATORY REQUIREMENTS 


What are ABS unit identifiers and why do | need to use them? 
How do! find out what the unit record identifiers are? 

Which datasets must include ABS unit identifiers? 

Will all CURFs contain the same unit record identifiers? 

What is a RADL audit? 

Are there any other mandatory requirements for using the RADL? 


What are ABS unit identifiers and why do | need to use them? 
The ABS unit identifiers are unique unit record identifiers that can be used to match the multiple levels of a 
dataset together. Common examples include: 


ABSHID ABS Person Identifier 

ABSFID ABS Family Identifier 

ABSIID ABS Income Unit Identifier 

ABSPID ABS Person Identifier 

ABSCID ABS All Conditions Identifier 

ABSRSID ABS Restrictions Identifier 

ABSSAID ABS Specific Activities Identifier 
ABSRCID ABS All Recipients Identifier 
ABSBAID ABS Broad Activities Identifier 
ABSAPID ABS Providers of Assistance Identifier 


The ABS unit identifiers also enable the automated checking facilities in the RADL to perform confidentiality 
checks, and assist in the automated clearance of your output. You must retain all ABS unit identifiers when 
running jobs and creating saved datasets containing unit record data. When jobs are run without retaining 
the ABS unit identifiers, this will cause the job to fail and it will not generate any output. 


How do | find out what the unit record identifiers are? 

Refer to the RADL documentation for each CURF you are using to see which ABS unit identifiers are used 
in that CURF. The documentation is in the 'CURF reference documentation’ link of the navigator of the 
RADL. 


Which datasets must include ABS unit identifiers? 

All datasets that you create that include unit record data must contain the ABS assigned unit identifier fields 
for that data. This includes your temporary (work) datasets. Do not drop unit identifier fields (i.e. if you only 
keep a subset of variables, the unit identifier fields must be in this subset). Do not rename these fields. 
Make sure you input these fields if you are reading a text data file version of a CURF. Saved files containing 
aggregate data only do not need to retain unit identifiers. 


Will all CURFs contain the same unit record identifiers? 

Unit identifiers are not always the same and may not always remain the same for different year versions of 
the same CURF. For example, the National Health Survey, 2001 contains two unit identifiers, while the 
National Health Survey, 2004-05 contains five unit identifiers. 


What is a RADL audit? 
Audits of jobs submitted to the RADL are conducted throughout the year by the ABS. The purpose of these 
audits is to identify confidentiality issues within users' code, log or output files. Confidentiality issues may 


arise through users failing to follow, accidentally or otherwise, restrictions or rules for using the RADL. 


For each job that is audited, the code, log and output files are manually inspected by ABS staff. Ifa 
confidentiality issue is identified, the user will be notified. Users are required to cooperate with any ABS 
directions in respect to their code and output. 


Are there any other mandatory requirements for using the RADL? 
To view all mandatory requirement for using the RADL see the Rules and restrictions for use section of this 
User Manual: ABS Remote Access Data Laboratory (RADL). 
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What does my job's status mean? 
What do | do if a job does not appear to be progressing? 


What does my job's status mean? 

Submitted to RunCasper: This is the status that all SAS jobs go to after they have been submitted while they 
are waiting to complete. If you find that your jobs are not progressing, see What do | do if a job does not 
appear to be progressing? 


Submitted to Qman: This status is the stage that SPSS and Stata jobs go to after they have been submitted 
while they are waiting to complete. If you find that your jobs are not progressing, see below: What do | do if a 
job does not appear to be progressing? 


Error: The Error status occurs when there is a system error during a job submission. If this occurs, try 
submitting your job again, or contact the RADL Administrator for assistance by emailing radl@abs.gov.au or 
call 02 6252 7714. 


Failed: The Failed status occurs when there has been an error in the submitted code. There are a few 
reasons why a job may fail, such as code error, failing to reference the CURF, spelling mistakes and failing to 
retain unit identifiers. Review the log file and your code. 


Timed out: The Timed out status occurs when your job has not completed within time limits. When submitting 
a new request within the RADL, you have the option of running it as a "Normal" job, which will run for up to 10 
minutes. If the job has not completed within 10 minutes, the job will be set to "Timed out" and will not finish. 
The second option is to run the job as a "Long" job, this will allow the job to run for longer than 10 minutes. If 
you have run your job as a "Normal" job and are experiencing timing out , it is recommended that you try 
running your job as a long job. An alternative to running a ‘long’ job is to split your code into several smaller 
jobs. You should also check your log file and your code. If you continue to have problems, contact the RADL 
Administrator for assistance by emailing radl|@abs.gov.au or call 02 6252 7714. 


Complete: The Complete status occurs when your job has completed without error. Note that a status of 
Complete does not necessarily mean the output has been generated (e.g. if your code does not ask for 
output). 


On Hold: The On Hold status occurs when generated output has been withheld for manual clearance. Output 
may be withheld due to exceeding file size or level of detail limits. If you receive the On Hold status, you can 
request your output to be released by clicking on the "Request Withheld Outputs” link on the job results page. 
For further information see the Requesting withheld output if the status of your RADL job is “On Hold” section 
of the User Manual: ABS Remote Access Data Laboratory (RADL). 


What do | do if a job does not appear to be progressing? 
If a normal job (not including long jobs) does not appear to be progressing after 10 minutes and has a status 
of ‘submitted’, you can: 


check your log file and code for mistakes 

check that you have retained all unit identifiers 

check for RADL alerts - occasionally ABS conduct routine maintenance that may affect RADL users 
try submitting your job again. 


If the job still doesn't complete, contact the RADL Administrator for assistance by emailing radl@abs.gov.au 
or call 02 6252 7714. 
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OUTPUT 


What do the different output types mean? 

How do | validate my results? 

Can I save my datasets in the RADL? 

How should | store output exported from the RADL? 

How do | produce aggregate data in Comma Separated Value format? 
Can the RADL output be produced in HTML format? 

Is graphical output available from the RADL? 


What do the different output types mean? 

There are several different categories of output files and there are different use and storage obligations for 
each. For further information see the Checking submitted jobs and output section of the User Manual: ABS 
Remote Access Data Laboratory (RADL). 


General release output: This output has low confidentiality risk and is considered safe for sharing or 
publishing. General release output files are attached in the Results section of your job, and can be saved in 
your own environment. 


Keep secure output: This output is too detailed for general release. Keep Secure output cannot be shared 
with other users unless they also have been approved by the ABS to access that CURF. Keep secure output 
files are attached in the Results section of your job and can be saved in your own environment. The Keep 
secure output must be stored securely or destroyed after viewing. For more information on how to store 
microdata securely see the Secure storage of ABS CURF microdata page of the User Manual: Responsible 
Use of ABS CUREFs. 


Withheld output: This output has been deemed risky and requires manual clearance by an ABS RADL 
Administrator. Withheld output can occur when output exceeds file size limits, contains a large number of 
single contributor cells or your code uses certain commands. Withheld output is not provided with the job 
results, but can be requested from the job results page. For further information see the Requesting withheld 
output if the status of your RADL job is “On Hold” section of the User Manual: ABS Remote Access Data 
Laboratory (RADL). Withheld output that is approved to be released is usually released as Keep secure 
output. 


Denied output: Output that has been denied will not be generated. This occurs when the output contains too 
much detail for the system to release. Denied output cannot be requested. 


How do | validate my results? 
The 'CURF reference documentation’ link in the navigator of the RADL contains frequency and weighting 
reports. These reports have been made available to assist users in validating their results. 


Can I save my datasets in the RADL? 

Datasets created in the RADL can be saved to your personal workspace in the RADL. For instructions, see 
How to save files to your personal workspace in this User Manual: ABS Remote Access Data Laboratory 
(RADL). Datasets cannot be saved outside your personal workspace (for example to your hard drive). 


How should | store output exported from the RADL? 
For information on the storage of microdata output see the Secure storage of ABS CURF microdata section 
of the User Manual: Responsible Use of ABS CUREFs. 


How do | produce aggregate data in Comma Separated Value format? 

The RADL has a method for producing Comma Separated Value (CSV) format output in all three languages. 
When producing these outputs, it is important to remember that output can and may be denied due to file size 
limits, So you may want to limit the amount you wish to output per job. 


SAS 

The %PrintCSV macro in SAS provides users with the ability to produce protected aggregate output in CSV 
format. The macro should be invoked by the user directly after creating the aggregate dataset by using, for 
example, Proc Freq or Proc Summary. The macro outputs these datasets in CSV format to the relevant 
output file. 


proc freq data=CURFLEVELL1 noprint; 
tables age*sex /out=out_table; 

run; 

%YprintCSVv; 


There is also an option to suppress printing of column headings: 


proc freq data=CURFLEVELL1 noprint; 
tables age*sex /out=out_table; 

run; 

YprintCSV(title=none); 


The "%PrintCSV" macro should be used instead of a data step which includes a ‘put’ statement: 


proc freq data=CURFLEVELLI noprint; 
tables age*sex/out=out_table; 

run; 

data _null_; 

set out_table; 

put sas ',’ sex ',' age ',’ count; 

run; 


To apply the "%PrintCSV" correctly remove the data step which includes the ‘put’ statement: 


proc freq data=CURFLEVEL1 noprint; 
tables age*sex/out=out_table; 

run; 

%PrintCSV 


SPSS 
The List command for SPSS will create output in a format similar to the CSV format but must only be used in 
conjunction with aggregated datasets. 


get file saved. AGGREGATEFILE. 
list VARIABLE1 VARIABLE2 VARIABLE3 VARIABLE4 VARIABLE5 VARIABLE6 VARIABLE7. 


STATA 
There is no equivalent function for Stata. 


Can the RADL output be produced in HTML format? 
The RADL does not support HTML output. RADL output is produced in txt format. See also How do | produce 
aggregate data in Comma Separated Value format?. 


Is graphical output available from the RADL? 
Graphical output not available in the RADL due to the risk of disclosure. 
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Why can't | see my approved CURFs in the RADL? 

| get an error message saying that the dataset can't be found 
| get an error message saying that the variable can't be found 
What do | do if a job does not appear to be progressing? 
What do | do if | am having coding problems? 

My output is being denied but I'm not sure why 

Why can't | use bootstrapping commands in RADL? 

What are Parser Errors? 


Why can't | see my approved CURFs in the RADL? 
There are three possible reasons: 


e Check that you have completed all steps in the Submit new request screen. For further assistance see 
the Submitting a request in the RADL section of the User Manual: ABS Remote Access Data 
Laboratory (RADL). 

e For some CURFs, additional RADL user IDs are allocated to approved users. Some users will have 
user identifiers ending with "_INNOV", "_STATE", "_DETAIL", or "_REM". If you have one of these user 
IDs you should log in with each of your user IDs to check whether the CURF you want to use is 
attached to your other ID. 

e Your application for access may not have been finalised yet . Check in MiCRO to see the status of your 
CURF application or contact microdata.access@abs.gov.au or call 02 6252 7714 for assistance. 


| get an error message saying that the dataset can't be found 
Check that the dataset is being referenced correctly. Information on library and dataset names is available in 
the 'CURF reference documentation’ link in the navigator of the RADL. 


If you are trying to access a saved dataset, check that the dataset exists. You can do this by using File 
management command code. 


| get an error message saying that the variable can't be found 

Check that the variable is being spelt correctly (especially check for underscores, and the case) and that the 
variable is available on the CURF being accessed. Information on variable names is available in the 'CURF 
reference documentation’ link in the navigator of the RADL. A common problem is that a variable may be 
available one level of the CURF but not another, for example on the household level file, but not the person 
level file. 


What do | do if a job does not appear to be progressing? 
If a normal job (not including long jobs) does not appear to be progressing after 10 minutes and has a status 
of ‘submitted’, you can: 


e check your log file and code for mistakes 

e check that you have retained all unit identifiers 

e check for RADL alerts - occasionally ABS conduct routine maintenance that may affect RADL users 
e try submitting your job again. 


If the job still doesn't complete, contact the RADL Administrator for assistance by emailing radlI@abs.gov.au 
or call 02 6252 7714. 


What do | do if !am having coding problems? 

The ABS does not provide a code advisory service however the ABS have produced introductory RADL 
tutorials for SAS, SPSS or Stata. The tutorials introduce users to the RADL system and use code to produce 
basic tables and statistics. The tutorials are available in the ‘Information for all RADL Users' link in the 
navigator of the RADL. 


Here are some common problems that users experience: 


e failing to reference the CURF properly 
e not typing variable names in upper case when using Stata. 


Common problems with SAS code: 


e missing, or extra semicolons - check that your semicolons are where they should be 

missing ‘by' statements when merging data sets 

merging levels within a CURF with variables that exist on both, but don't always take the same value on 
both 

e performing character operations with numeric variables, or vice versa 

e attempting to use banned commands. 


Common problems with Stata code: 


e attempting to use user-written packages which have not been added to the RADL 
e attempting to use banned commands (particularly svrtab) 
e excessively long lines of code (use continuation character "///" and start a new line). 


Check your job's log file to see where the errors are occurring within your code, this will help determine where 
your code is going wrong. 


My output is being denied but I'm not sure why 

Output may be denied due to exceeding file size or level of detail limits. You can try splitting your code or 
output into smaller sections. While this is allowed for aggregate output, note that cumulatively downloaded 
unit record level output is not allowed and is audited. 


RADL SPSS sometimes denies output and does not print an appropriate error message as a result of code 
references to non-existent variables. Check the spelling of all variables (especially underscores) and re-run 
your job. For further information about variable names, see the data item list in the 'CURF reference 
documentation’ link in the navigator of the RADL. 


Why can't | use bootstrapping commands in RADL? 
Bootstrapping commands are banned in RADL. 


To use bootstrap commands appropriately a user needs to be able to identify the Primary Selection Units 
(PSU). In the case of ABS CURFs, all respondents in the same Collection District (CD) belong to the same 
PSU. However, grouping respondents into PSUs significantly increases the risk of a respondent being 
identified so the ABS cannot release this information. To enable researchers to produce accurate variance 
estimates, the ABS releases the replicate weights which take the sample design into consideration, as well as 
any other steps in the estimation process such as non-response adjustments. 


For Stata users, the ABS has made available the user-supplied svr suite of commands to assist users in 
calculating standard errors using replicate weights. See the How can | use the replicate weights in analysis or 
estimation of sampling error? section of the Survey and replicate weights section of the User Manual: ABS 
Remote Access Data Laboratory (RADL). 


What are Parser Errors? 

Parser errors occur when a conflict arises between a user's code and the built-in protections of the RADL and 
usually occurs in conjunction with the use of ‘Array’ statements. To fix a parser error you will need to alter the 
code. Additionally the following words are considered 'keywords' in SAS and can't be used as variable names 
in the RADL. 

AND ELSE LE MODIFY SAME 

BETWEEN END LIKE NE SELECT 

CALL EQ LINES NOT SET 


CARDS FILE LINES4 


CARDS4 GE LOG 
CONTAINS GT LT 

DATA IF MAX 
DATALINES _ IN MERGE 
DATALINES4 IS MIN 

DO LAST MISSING 
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SAMPLE CODE AND USING CURFS 


How do | generate a cross-tabulation? 

How do | create ranges in my data? 

How do | save a dataset | have created from a CURF? 

Can | merge two levels within a CURF? 

Can I merge two CURFs? 

What is the difference between unit record data and aggregate data? 


How do | generate a cross-tabulation? 
The following are examples of how to run a cross-tabulation in each language: 


SAS 

proc freq data=LIBNAME.CURFLEVEL1I ; 

table VARIABLE1*VARIABLE2*VARIABLES ; 

format VARIABLE1 FORMAT1. VARIABLE2 FORMAT2. VARIABLE3 FORMATS. ; 
weight WEIGHT; 

run; 


SPSS 

get file=CURFLEVEL1. 
weight by WEIGHT. 
execute. 


crosstabs 

/tables=VARIABLE1 by VARIABLE2 by VARIABLE3 
/formats=avalue tables 

/cells=counts. 


Stata 
use "CURFLEVEL1 "™ 
table VARIABLE1 VARIABLE2 VARIABLE3 


How do | create ranges in my data? 
The following are examples of how to create ranged data in each language using the age variable: 


SAS 
You can create ranged data within SAS by using the Proc Format function. 


proc format; 


value agef 

low - 14 ='under 15' 
15 - 24 = '15-19' 

25 - 34 = '25-34' 

35 - 44 = '35-44' 

45 - 54 = '45-54' 

55 - 64 = '55-64' 

65 - 74 ='65-74' 


75 - 84 = '75-84' 


85 - high = '85+": 


proc freq data = LIBNAME.CURFLEVEL1; 
table age; 

format age agef. ; 

run; 


SPSS 
You can create ranged data within SPSS by using the Recode function. 


get file=CURFLEVEL1. 
execute. 


recode AGE (LO thru 14=1) (15 thru 24=2) (25 thru 34=3) (35 thru 44=4) 

(45 thru 54=5) (55 thru 64=6) (65 thru 74=7) (75 thru 84=8) (85 thru HI=9) into AGEGRP. 
variable labels AGEGRP ‘Age Group’. 

execute. 

value labels AGEGRP 1 ‘under 15’, 2 '15-19', 3 '25-34', 4 '35-44', 

5 '45-54', 6 '55-64', 7 '65-74', 8 '75-84', 9 '85+'. 


frequencies variables=AGEGRP 
/statistics=minimum maximum mode 
/order=analysis. 


Stata 
You can create ranged data within Stata by using the Recode function. 


use "CURFLEVEL1" 


generate agegroup = age 

label variable agegroup "Age Group” 

recode agegroup (0/14=1) (15/24=2) (25/34=3) (35/44=4) (45/54=5) (55/64=6) (65/74=7) (75/84=8) 
(85/150=9) 

label define agegrouplbl 1 "under 15" 2 "15-19" 3 "25-34" 4 "35-44" 5 "45-54" 6 "55-64" 7 "65-74" 8 "75-84" 9 
"85+", modify 

label value agegroup agegrouplbl 

list agegroup in 1/10 


How do |! save a dataset | have created from a CURF? 

You can save your work in the RADL and then use your saved dataset in a later job. Example code showing 
you how to save datasets and use saved datasets is in the How to save files to your personal workspace 
section of the User Manual: ABS Remote Access Data Laboratory (RADL). 


Can | merge two levels within a CURF? 

You can merge multiple levels of a CURF. This is done by sorting each level by a common ABS unit identifier, 
then merging by the same identifier. Merged datasets can be saved in your personal workspace in the RADL 
for later use. For more information about saving, see the How to save files to your personal workspace 
section of the User Manual: ABS Remote Access Data Laboratory (RADL). 


Below are some examples on how to merge two levels within a CURF, using SAS, SPSS or Stata. 


SAS: 

proc sort data = LIBNAME.CURFLEVEL1 out = psndata; 
by ABSHID; 

proc sort data = LIBNAME.CURFLEVEL2 out = hhdata; 
by ABSHID; 


data SAVED.MERGED; 
merge psndata hhdata; 
by ABSHID; 

run; 


SPSS: 

get file CURFLEVEL1. 

sort cases by ABSHID. 

get file CURFLEVEL2. 

sort cases by ABSHID. 

match files file=* 
/table=CURFLEVEL1 

/by ABSHID. 

execute. 

save outfile = SAVED'MERGED'". 


Stata: 


use "CURFLEVEL1" 
sort ABSHID ABSIID ABSPID ABSFID 
save "SAVED'XTRAVARS'", replace 


use " CURFLEVEL2" 
sort ABSHID ABSIID ABSPID 
save " SAVED'NEWVARS", replace 


use " SAVED'NEWVARS" 
merge ABSHID ABSIID ABSPID using ""SAVED'XTRAVARS" 
save "SAVED'MERGED", replace 


Can | merge two CURFs? 
You can merge two CURFs. However, you may only merge aggregate data to unit record data, or aggregate 
to aggregate data between the CURFs. You must not merge unit record to unit record data across CURFs. 


To merge CUREFs, in the New Request screen, select more than one CURF in the Microdata window by 
holding down the Control button on your keyboard. You will also need to reference both CUREFs in your code. 
Alternatively your code can reference a saved dataset from one CURF and reference another CURF that you 
have selected in the Microdata window. Note that not all CURFs will have the same variable names, so you 
may need to use renaming commands within your code. Further information about referencing in your code is 
available in the How to reference the CURF in your code section of the User Manual: ABS Remote Access 
Data Laboratory (RADL). 


What is the difference between unit record data and aggregate data? 

Unit record data lists individual values of variables for the record. Aggregate data collapses variables across 
multiple records. Aggregate data examples include means, medians, ranges or regressions. Unit record data 
includes individual records from the CURF. Greater protections are placed on unit record data and can limit 
the output you are able to receive. With aggregate data, the greater the level of aggregation, the greater 
amount of data you can output. Note that when working with unit record data, ABS unit identifiers must be 
retained. For further information see Retention of ABS identifiers section of the User Manual: ABS Remote 
Access Data Laboratory (RADL). 
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How should | use survey weights in my model? 

How do | use normalization with SAS, SPSS and Stata? 

How can | use the replicate weights in analysis or estimation of sampling error? 
What parameters do | use with the svr suite of commands in Stata? 

Where can | find useful references on the use of survey weights in modelling? 


How should | use survey weights in my model? 

Survey weights are most commonly used to produce estimates of aggregates, means, and quantiles for a 
population. The use of weights for more complex survey analysis is less clear-cut. There is no one unified 
view by statisticians on if, and how, weights should be used when undertaking a modelling exercise. 


The decision whether to use weights or not could have an effect on point estimates produced from the model, 
not only in terms of magnitude but possibly also in terms of sign and significance. Standard errors of these 
estimates will also differ in magnitude depending on whether weights are used. In general, use of survey 
weights in the model should lead to accurate point estimates. However, the standard errors of these 
estimates will not tend to be as accurate. 


One approach that may help in partially addressing the issue of less accurate standard errors is to first 
normalize the weights. Normalizing of weights involves multiplying the weight of each person by a constant 
factor so that they add up to a desired value. For the purposes of survey analysis, it is common to normalize 
the weights to add up to the sample total. It is suggested that normalized weights be calculated at state level 
where possible due to the different sampling fractions that are used in each state. 


State-level normalized weights can be calculated by multiplying the person weight of an individual within state 
h by the proportion of people in the state that were in the sample: 


“ 2, 
Wy, = We, |— 
if ih N. 

& 


where 

Ww, 1s the normalized weight for unit 7 in state 2 

Ww, 1s the original person weight for unit? in state 2 
#, 1s the sample total in state ” 

WV, is the population total in state # 


How do | use normalization with SAS, SPSS and Stata? 
Support for normalized weights in modelling varies for each of the different software packages available in 
the RADL. 


SAS 

When using the LOGISTIC or PHREG procedures, users may specify an option /NORMALIZE when using 
the ‘weight’ statement. This will cause the weights specified by the ‘weight’ statement to add to the total 
sample size. Note that this option does not necessarily produce normalized weights at state level. 


SPSS 
Minimal support for normalized weights. 


Stata 
When producing some models, Stata will rescale the specified weights to add to the total sample size. 


How can | use the replicate weights in analysis or estimation of sampling error? 

While SAS v9.1 provides the PROC SURVEYMEANS procedure and Stata v10 provides the svy suite of 
commands, these facilities do not use replicate weights. Rather than using replicate weights these facilities 
require identification of both stratification and clustering. To help protect against disclosure of individuals, 
these details have not been included on the CURF. The user written svr suite of procedures is available for 
RADL Stata users. These procedures are made available but are not supported by the ABS. 


What parameters do | use with the svr suite of commands in Stata? 

The methodology used should be set to /k1: 

svrset set meth jk1 

The ABS uses the delete 1 group jackknife method rather than the unstratified delete 1 jackknife method that 
Jk1 specifies. However this setting, when used in conjunction with the other parameters below, produces 
suitable estimates of variance of estimates. 


The other parameters will vary depending on the variable names for the primary and the replicate weights 
and the number of replicate weights. For example, if the primary weight on the CURF was ‘PRSNWGHT’, and 
if there were 30 replicates groups and the replicate weights were named RW1 to RW30, then the remaining 
parameters would be set using: 

svrset set pw PRSNWGHT 

svrset set rw RW* 

svrset set dof 29 

Note that the dof parameter should be set to the number of replicate groups minus one. 


Refer to the technical information for further information about the weights on your CURF, via the 'CURF 
reference documentation’ link in the navigator of the RADL. 


A useful summary of replicate weights is available in the Stata help documentation: http:/Awww.ats.ucla.edu/ 
stat/stata/Library/replicate_weights.htm 


Where can | find useful references on the use of survey weights in modelling? 
The following references may assist your decision on whether or not to use survey weights for your particular 
purpose. They have been selected as a cross-section of discussion on the issue of using weights in models. 


Chambers, R.L. and C.J. Skinner (eds.) (2003), Analysis of Survey Data, Chichester: Wiley. 
DuMouchel, W.H. and G.J. Duncan, (1983) "Using Sample Survey Weights in Multiple Regression Analyses 
of Stratified Samples", Journal of the American Statistical Association, Vol. 78, No. 383. (September), pp. 


535-543. 


Magee, L., Robb, A.L., and J. B. Burbidge, (1998), "On the use of sampling weights when estimating 
regression models with survey data”, Journal of Econometrics, Volume 84, Issue 2 (June), Pages 251-27.1 


Pfeffermann, D. (1993), "The Role of Sampling Weights When Modeling Survey Data", International 
Statistical Review, Vol. 61, No. 2. (August), pp. 317-337. 


Pfeffermann, D (1996), "The Use of Sampling Weights for Survey Data Analysis", Statistical Methods in 
Medical Research, 5, pp. 239-261. 


Skinner, C.J., Holt, D., and T.M.F Smith (1989), Analysis of Complex Surveys, Chichester: Wiley. 


Winship, C., and L. Radbill (1994), "Sampling Weights and Regression Analysis." Sociological Methods and 
Research 23(2):230-257. 
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RADL has help information located in the left navigator of the RADL via the links 'CURF reference 
documentation’, ‘Information for all RADL users' and 'ABS alerts’. These provide a range of information 
designed to help RADL users, including: 


e general information for getting started with SAS, SPSS and Stata 
e CURF referencing information including library references for the each level of each CURF 
e CURF frequency and weighting reports to assist you in validating your results 


links to CURF technical manuals, providing information regarding the data items and variables for each 
CURF 


e alert documents providing information about any RADL outages. 


RADL users should also use the log files to help resolve problems in obtaining output. Further information is 
provided in the Errors and problems section of this manual. 


If you encounter any problems accessing or using the RADL, email the RADL Administrator at 
radl|@abs.gov.au or phone 02 6252 7714. 
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This User Manual provides ABS Remote Access Data Laboratory (RADL) users and potential users with 
information about confidentialised unit record files (CURFs). It describes RADL functionality and supporting 


processes, and details rules for using commands in the three analytical languages supported (SAS, SPSS 
and Stata). 


This page last updated 16 August 2012 


© Commonwealth of Australia 2014 
per __| 


Unless otherwise noted, content on this website is licensed under a Creative Commons Attribution 2.5 Australia Licence together with any terms, conditions and 
exclusions as set out in the website Copyright notice. For permission to do anything beyond the scope of this licence and copyright terms contact us. 


Tear 


Australian! AUStralian Bureau of Statistics 


Statistics 


1406.0.55.002 - User Manual: ABS Remote Access Data Laboratory (RADL), 
Mar 2006 


Latest ISSUE Released at 11:30 AM (CANBERRA TIME) 22/02/2012 Reissue 


22/02/2012 - Information held in this manual has been updated to include updated technical information on 


the RADL, more detail on the roles and responsibilities of RADL users and a comprehensive list of RADL 
Frequently asked questions 


Formerly: ABS Remote Access Data Laboratory (RADL): User Guide. 


17/08/2012 - Changes to this user manual have been made to reflect changes made to the RADL home page 
interface. 


This page last updated 16 August 2012 


© Commonwealth of Australia 2015 


a) 
Unless otherwise noted, content on this website is licensed under a Creative Commons Attribution 2.5 Australia Licence together with any terms, conditions and 
exclusions as set out in the website Copyright notice. For permission to do anything beyond the scope of this licence and copyright terms contact us. 


